{
    title:  "Plugins",
    crumbs: [ 
        { "Developer's Guide": "index.html" },
    ],
}
            <h1>Creating Plugins</h1>

			<p>Plugins are a special type of package that provides extended processing for the Expansive 
			pipeline. Plugins provide <em>services</em> that are used to transform a file from one format 
			to another. For example: the <em>exp-md</em> plugin provides the <em>compile-markdown-html</em> 
            service that transforms Markdown files with a <em>.md</em> extension into HTML files with a 
            <em>.html</em> extension.</p>

            <h2>Plugin Structure</h2>
            <p>Plugins are delivered as <a href="http://embedthis.com/pak">Pak</a> packages. They always contain 
            a minimum of two files:</p>
            <ul>
                <li>package.json &mdash; that describes the package</li>
                <li>expansive.es &mdash; that implements the plugin</li>
            </ul>

            <p>The <em>package.json</em>file contains the name of the plugin, the current version and the list
            of dependent packages required for the plugin.</p>
            <p>The <em>expansive.es</em> file is a Javascript file that typically contains just one call to 
            <em>Expansive.load</em> to define the services provided by the plugin. For example:</p>
            <code class="inverted">Expansive.load({ 
    services: [ {
        name: 'my-markdown',
        options: 'my configuration options',
        init: function(transform) {
        },

        transforms: [ {
            name: 'compile',
            mappings: {
                'md': 'html'
            },
            init: function(transform) {
            },
            resolve: function(path) {
                return path
            },
            render: function(contents, meta, transform) {
                return contents
            },
            pre: function(transform) {
            },
            post: function(transform) {
            },
        } ]
    } ]
})</code>

            <p>The plugin calls <em>Expansive.load</em> and provides a <em>services</em> array of one or more service definitions. Each service is a hash of properties that contains a <em>name</em> and a set of transforms. The transforms applied to specific content <em>mappings</em> that defines the file types for which the transform applies.</p>
            <p>Transforms can provide a set of callback functions that will be invoked by the Expansive transformation pipeline as required. A transform may provide a <em>name</em> property that is appended to the service name to create a unique transform name. If omitted, the name of the service is used for the transformation name.</p>

            <h2>Expansive Pipeline</h2>
            <p>The Expansive pipeline calls plugin services to transform files from one format to another. It does this by calling a transform <em>render</em> function and providing it with the current file contents, meta data and service configuration. The order of transformations is determined by the content filename extensions. For example, consider the file:</p>

            <code class="inverted">index.html.md.exp</code>
            <p>This file uses embedded Javascript (<em>exp</em>) that is run to generate a Markdown file 
            (<em>md</em>), that is to be converted to HTML (<em>html</em>). Note extensions are read right to left.
            To implement this, the pipeline will use the transformations:</p>
            <ul>
                <li>exp &rarr; md</li>
                <li>md &rarr; html</li>
            </ul>

            <p>The pipeline performs these transformations by invoking the plugin service transforms have the 
                requisite mappings:</p>
            <ul>
                <li>exp</li>
                <li>markdown</li>
            </ul>

            <h3>Null Transforms</h3>
            <p>Plugins can also define a service for that maps a file type to itself. For example: <em>html</em> &rarr; 
            <em>html</em>. This "null" transformation can be used for the last extension in a filename to perform
            final stage processing when the contents are mutated, but the file type does not change. For example: 
            to minify a script file.</p>

            <h2>Mappings</h2>
            <p>A service defines the file types for which it should apply via the <em>mappings</em> property.
            The mappings takes the following format:</p>
            <code class="inverted">mappings: {
    'from': 'to',
    'from': [ 'to' ],
}</code>
            <p>The <em>from</em> property is the original (outer-most) file extension. The <em>to</em> property is 
            the destination (inner) file extension. It may also be an array of destination extensions. Expansive supports
            two abbreviated mappings syntax. Mapping entries may be set to a single string if the <em>from</em> and
            <em>to</em> values are the same. The Mappings property may be also set to a file extension string if 
            only one extension is supported and the <em>from</em> and <em>to</em> file extensions are the same. For 
            example:</p>
            <code class="inverted">mappings: 'html'

mappings: {
    'html'
}</code>

            <h2>Init Function</h2>
            <p>A service may define an <em>init</em> function to be invoked before processing commences. A transform may also define an init function for a similar purpose.</p>

            <h2>Resolve Function</h2>
            <p>Some transformations may need to alter the destination filename. For example, a Javascript minifier may change the output filename by using a <em>.min.js</em> extension. The resolve function is invoked to determine the final destination filename for the contents. The filename may be prefixed with a "|" (pipe) character to signifiy that this is the end of the pipeline processing and not more analysis of the filename extensions should take place. Returning null signifies that this content is not required.</p>

            <h2>Render Function</h2>
            <p>The <em>render</em> function is invoked to transform the file contents. It is passed three arguments:</p>
            <ul>
                <li>contents &mdash; The current file contents as an in-memory string.</li>
                <li>meta &mdash; Meta data properties for the file.</li>
                <li>transform &mdash; The transform data object containing a reference to the service configuration.</li>
            </ul>
            <p>The render function should process and return the new contents. The transform can instruct the pipeline
            to delete the file by returning null.</p>

            <h2>Pre Function</h2>
            <p>If a transform defines a <em>pre</em> function, it will be invoked before all pipeline processing. 
            This "pre-processor" is useful to configure the environment or dynamically create pages.
            It is passed two arguments:</p>
            <ul>
                <li>meta &mdash; Meta data properties for the file.</li>
                <li>transform &mdash; The transform data object containing the service configuration.</li>
            </ul>

            <h2>Post Function</h2>
            <p>If a transform defines a <em>post</em> function, it will be invoked after all pipeline processing to 
            perform "post" processing. It is passed two arguments:</p>
            <ul>
                <li>meta &mdash; Meta data properties for the file.</li>
                <li>transform &mdash; The transform data object containing the service properties.</li>
            </ul>

            <h2>Using Meta Data</h2>
            <p>Expansive defines various meta data properties for the original filename, destination filename
                and various other useful path properties.</p>

            <table class="ui celled table" title="properties">
                <thead>
                    <tr><th>Property</th><th>Description</th></tr>
                </thead>
                <tbody>
                    <tr><td>source</td><td>Current source file being processed. Relative path including 
                        <em>contents</em>, <em>layouts</em> or <em>partials</em> directory prefix.
                        For example: <em>contents/sub/index.html</em>.</td></tr>
                    <tr><td>sourcePath</td><td>Current source file being processed. Relative path excluding 
                        <em>contents</em>, <em>layouts</em> or <em>partials</em> directory prefix.
                        For example: <em>sub/index.html</em>.</td></tr>
                    <tr><td>dest</td><td>Destination filename being created. Relative path including 
                        <em>dist</em>. For example: <em>dist/sub/index.html</em>.</td></tr>
                    <tr><td>destPath</td><td>Destination filename being created. Relative path excluding 
                        <em>dist</em>. For example: <em>sub/index.html</em>.</td></tr>
                    <tr><td>base</td><td>Source file without <em>contents</em> or <em>lib</em>. For example: 
                        <em>sub/index.html</em>.</td></tr>
                    <tr><td>document</td><td>Source of the document being processed. For partials or layouts, it is 
                        set to the invoking document. For example: <em>sub/index.html</em>.</td></tr>
                    <tr><td>url</td><td>Url reference made from <em>path</em>. For example: 
                        <em>sub/index.html</em>.</td></tr>
                    <tr><td>top</td><td>Relative URL to application home page. For example: <em>../</em>.</td></tr>
                </tbody>
            </table>

            <p>See <a href="../users/meta.html#properties">Meta Data</a> for the full list of meta data properties.</p>

            <h3>Installing Plugins</h3>
            <p>Some plugins need to rely on external programs. If these external programs are delivered via
            <a href="http://embedthis.com/pak/">Pak</a> packages, then the plugin can simply specify the other 
            package in its dependency list. If the plugin needs to configure the environment or run other 
            external commands before it can run, then some installation setup may be needed.</p>

            <p>The plugin <em>package.json</em> file may specify a script to run once the plugin package is installed.
            This script can then perform any required plugin configuration. Packages can "hook" two key package events:</p>
            <ul>
                <li>postcache &mdash; run after the package is installed in the ~/.paks package cache</li>
                <li>install &mdash; run after the package is locally installed under ./paks</li>
            </ul>
            <p>For example, the <em>exp-js</em> package.json file contains the following event script:</p>
            <code class="inverted">"scripts": {
    "postcache": {
        "script": "Cmd.locate('uglifyjs') || run('npm install -g uglifyjs')" 
    }
}</code>
            <p>This will try to locate the <em>uglifyjs</em> command and if not found, will use <em>npm</em> to
            install it globally.</p>

            <h3>Plugin Configuration</h3>
            <p>The Plugin services may define default configuration in their service definition that may then be
            overridden by user configuration in the top level expansive.json. 
            A service may define any custom property in the service transform definition. For example:</p>
            <code class="inverted">Expansive.load({ 
    services: {
        name: 'minify-html',
        options:  '--remove-comments'
        ...
    }
})</code>
            <p>The <em>options</em> property defines the default HTML minification options. The user can then
            override in the <em>expansive.json</em> via:</p>
            <code class="inverted">{
    services: {
        'minify-html': {
            options: '--remove-comments --remove-attribute-quotes'
        }
    }
}</code>
            <p>Expansive creates an <em>enable</em> property for all services so the service does not need to 
            explicitly create it.</p>

            <h2>Example</h2>
            <p>Here is a sample plugin that transforms shell scripts and captures their output. This plugin
            implements the <em>shell</em> service and will handle files with a <em>.bash</em> and <em>.sh</em>
            extension.</p>
            <code class="inverted">Expansive.load({
    services: {
        name: 'shell',
        transforms: {
            mappings: {
                'bash',
                'sh'
            },
            render: function(contents, meta, transform) {
                return run(file.extension, contents)
            }
        }
    }
})</code>
            <p>Here is a sample plugin that implements the <em>compress</em> service to post-process
            files using <em>gzip</em>.</p>

            <code class="inverted">Expansive.load({
    services: {
        name:   'compress',
        <b>files:  [ '**.html', '**.css', '**.js' ]</b>,
        transforms: {
            post: function(transform) {
                let gzip = Cmd.locate('gzip')
                if (!gzip) {
                    trace('Warn', 'Cannot find gzip')
                    return
                }
                for each (file in directories.dist.files(<b>transform.service.files</b>, {directories: false})) {
                    file.joinExt('gz', true).remove()
                    Cmd.run('gzip ' + file, {filter: true})
                }
            }
        `
    }
})</code>
            <p>In this example, the plugin access the service <em>files</em> property for the set of file patterns
            to match.</p>

            <h2>Plugin Scripting</h2>
            <p>Expansive provides full access to the entire <a href="http://embedthis.com/ejscript">Ejscript</a>
            Javascript language. See the <a href="http://embedthis.com/ejscript/doc/">Ejscript API Documentation</a>
            and <a href="http://embedthis.com/ejscript/doc/ref/api/ejscript/index.html">Ejscript Script Library</a>
            for full details.</p>

            <h3>Expansive Convenience Functions</h3>
            <p>Expansive provides several convenience routines to make writing plugins easier.</p>

            <table class="ui celled table" title="functions">
            <thead>
                <tr><th class="eight wide column">Syntax</th><th>Description</th></tr>
            </thead>
            <tbody>
                <tr>
                    <td>addItems(collection: String, items: String|Array)</td>
                    <td>Add items to the named collection.</td>
                </tr>
                <tr>
                    <td>getFiles(patterns: String|Array, query: Object): Array</td>
                    <td>Query the meta data for matching files and return list of matching filenames.</td>
                </tr>
                <tr>
                    <td>getFileMeta(path: String): String</td>
                    <td>Get the file meta data for the given path.</td>
                </tr>
                <tr>
                    <td>getItems(collection: String): String</td>
                    <td>Return the items for a named collection.</td>
                </tr>
                <tr>
                    <td>removeItems(collection: String, items: String | Array)</td>
                    <td>Remove items from the named collection</td>
                </tr>
                <tr>
                    <td>trace(tag: String, ...args)</td>
                    <td>Emit trace to the console.</td>
                </tr>
                <tr>
                    <td>vtrace(tag: String, ...args)</td>
                    <td>Emit trace to the console if expansive is run in verbose mode.</td>
                </tr>
                <tr>
                    <td>run(cmd: String, contents: String): String</td>
                    <td>Run the command with the contents as standard input and return the output of the command.</td>
                </tr>
            </tbody>
            </table>

            <h2>Popular Plugins</h2>
            <p>Here are some sample plugin implementations</p>
            <ul>
                <li><a href="http://github.com/embedthis/exp-babel">exp-babel</a> &mdash; Babel for ES6 support.</li>
                <li><a href="http://github.com/embedthis/exp-css">exp-css</a> &mdash; CSS management.</li>
                <li><a href="http://github.com/embedthis/exp-gzip">exp-gzip</a> &mdash; File compression.</li>
                <li><a href="http://github.com/embedthis/exp-js">exp-js</a> &mdash; JS file management.</li>
                <li><a href="http://github.com/embedthis/exp-less">exp-less</a> &mdash; Less stylesheet processing.</li>
                <li><a href="http://github.com/embedthis/exp-markdown">exp-markdown</a> &mdash; Markdown file processing.</li>
                <li><a href="http://github.com/embedthis/exp-sass">exp-sass</a> &mdash; Sass stylesheet processing.</li>
                <li><a href="http://github.com/embedthis/exp-shell">exp-shell</a> &mdash; Shell script processing.</li>
            </ul>
